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Gerald H. Whitlock, Ph.D. " — 
University of Tennessee^ 

This paper is co^icerned With the problems associated with the setting 
of ^passing s^cores on professional licensing and certification examinations, 
.^.ince empirical -studies of the criterion related validity of ^these kinds of 
tests ane practically non-existent, no useful'^data are available by whicji 
"to relate successful and" unsuccessful test performance to successful and 
unsuccessful professional practice.' And until one obtains the data des- 
cribing t\re regression of job-performance on test performance, the setting . 
of cutting scores or passing levels will have an element of arbitrariness. 
However, data from othfrr performance domains which suggest that some tests 
have differen^al validity for minority groups, along with the faft that 
in some states professional licensing examinations have been shown to have 
an adverse impact on the menfcers of minority groups., together suggest that 
liceinsing examinations, even' if they do have a degree of oyer-all validity, 
may in fact have differential validity- for minority groups. • This would 
then^eem to east the problem 'in terms of the EEOC ^guidelines for compliance 
with Title VII of the 1964 Civil Rights Act since minority group members are 
being excluded from the practice of their profession in disproportionate 
numbers in the absence of any data indicating criterion related validity 
of the licensing examinations^or selection into the profession, and it is 
not known whether the scpres assigned to minority group members have the 
same predictive properties, from the stand point of competence to practice, 

as those assigned to majority group members, 
s 



Opinions differ as to whether U is feasible, or Indeed possitjle, to 
conduct criterion related validity studies in any of ttie professions. The 
problem of acquiring meaningful measures of job performan^ i^ cited as the 
difficulty. Such a position is made more difficult to defend in view of 
the fact that performance evaluations are rolitinely conducted in the profes- 
sions of engineering, accountirr^, teaching and law. Peterso^n has described 
a study in North Carolina where 90^ reliability was obtained for ratings 
of performance oftmedical practitioners. Even so, it does not- appear 
likely that any such studies are going to be conducted in the near future 
for the purpose of setting passing scores on^licensing examinations, so 
consideration needs to be given to some alternati\Aes which at least in 
soitie sense minimize arbitrariness. . ^ 

' r 

Current practice is to rely on content validity of profess^nal 
examinations. There are marked differences among the professionsToF 

A- 

measuring content validity and the quality of the procedures ranges from 
sloppy to excellent. Even in the same profession there may be wide differ- 
ences in methods across states and with the same range of quality. 

In any case the means are assumed to produce' the desired ends, so the 
examinations are administered at prescribed times and candidates who do-not 
reach pre-determined levels (a sqore of 70 off a 100 point scale seems to be 
favored) are not admitted to practice. In some cases, as in CPA exams, 
parts may be repeated and in others the entire exam must be- repeated, e.g.,^ 
law. The concern here is with determining the passing score. First are 
considered some typical methods foV accomplishing this (rememberijig that 
the following methods are used in the absence of any criterion related 
s vali dity data) . 

1 ' >«> 
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Use of a Norip Group \ 

A. This method uses an appropriate norm grpup, and sets the p,assing ^ 
score in terms of their performance. For example, one of the professions 
in Illinois admits candidates to the exam who are rather heterogeneous with 
respect to educa^onal achievement. Thus they have found that passing every 
one whose score is ajDove one standard deviation bejow th^ mean passes about 
the "right" proportion of can^datas tvery few of the Candidates with Ph.D's 
score that low) . - , . . 

*. B. Where only those candidates are admitted to the ex^ who have 
graduated from an accredited (by the professiofj) school, then one might go 
two standard deviation^ below the mean ^o s^t the passing score. 'Jhis . 
method assumes that a very sml^l 1 percentage of such graduates should fail. 

Absolute Gradingi . > 

In this method a predetermined passing score is set. Typically the 

score is 70 or 75, and anyone scoring^^ below that is failed. Ih\s is the 

typical method when the exam is essay or'p/oblem solving, e.g,, law, part 

of engineering, accounting, medicine. In some cases borderline cases are 

re&raded, in others npt. In some cases great ca-re is taken to achieve 

reliability of grading, in others almost none. The writer knows of several 

.states in which parts of the ess^y exams of a major profession aire scored 

by only one individual. As might be expected, such a procedure results in 

considerable variations from year ta-year in the number of candidates 

admitted to the profession. There are some obvious ways to impr^ove the 

« 

reliability of scores, e.g., multiple graders, use of model answers, 
and th^like, and given the test reliabi 11 ty , J t Is then prossibl^to be 
sure that with any desired probability the passing score actually used 
would not exclude someone whose true achievement was at the level of the 



nominal passing score (e.g., if the nominal passing score was 70, then a ^ 
passing score 1.645 standard errors of measurement below 70 would provide j 
95% confidence -that failing candidates would not have been passed). 

It can be seen that the major element of arbitrariness in all of the 
above methods is the setting of passing scores. Even the setting of parti-*. 

% i. 

cular confidence limits is arbitrary, although bolstered by tradition in 
some contexts. This arbitrariness. wil 1 remain in the absence of empirical 
validation data.. However, the writer believes that meanwhile for some of 
the professional exams and parts of others, the setting of passing scores 
tan be put on firmer grounds, both logically and statistically. The basic 
assumption of the method proposed is that faculty of accredited professional 

schools in cooperation with the accreditors provide the prop^ professional 

\ 

instruction and maintain proper staadards of accomplishment to insure that 
their graduates are competent to practic^ (n the .profession. The mechanisms 
of the method are as follows: ^ 

1. For all examinees for a particular examination administration \ 
who are recent graduates of an accredited school within the 
state, determine the regression equation for predicting exam- 

. illation sco\es from professional school Grade Point Average. 
(Y = a + b(GPA)) V . ^ J 

2. Calculate the predicted examination score for the lowest GPA' 
required to graduate. This is the nominal passing score .'^ 

3. Subtract 1.645 standard errors of estimate from the nominal 
passing score to arrive at the actual passing score. Thus 
one is 95% confident that an individual who failed should 
not have been pished. 
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This method applies where the examination is takefr, or those parts of 
it, before or soon after ygraduati on , e.g., bar exam. Other methods would 
need to be used for thosj^ parts of the exam which are given only after 
some number of years in practice, e.g., some certifying exams. 

Using the above method insures (wi^fh high probability) that anyone - 
^ who is admitted to practice has at least the competence of a graduate of 
an accredi ted schpol . " . 

Also, in the event that the exam scores and GPA correlated differently 
for members of minority groups, the separate regression equations could be 
used. ' , ' 

I 

* Some Unresolved Questions , 

Many states have more than one accredited school in a particular 
^ profession so the question arises whicJ one to*use, since the equations 

will differ along with minimum GPA's for graduation'. There would seem to 
- be some merit in using the official state school to ^et the standard.. How- 
ever, other accredited schools may then suffer if they attract less able 
students. Or, it could be argued that the equation should be based on tt^e 
\ ' __^school yielding the lowest passing score since this score does reflect a 
level of achievement that presumably meets accreditation standards. A 
compromise would be to use separate equations to determine the passing score 
for each school, and use their unweighted mean passing score as the official 
passing score. 
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